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Abstract.  Approximation  is  central  to  many  optimization  problems  and  the  supporting  theory  pro¬ 
vides  insight  as  well  as  foundation  for  algorithms.  In  this  paper,  we  lay  out  a  broad  framework  for 
quantifying  approximations  by  viewing  finite-  and  infinite-dimensional  constrained  minimization  prob¬ 
lems  as  instances  of  extended  real-valued  lower  semicontinuous  functions  defined  on  a  general  metric 
space.  Since  the  Attouch-Wets  distance  between  such  functions  quantifies  epi-convergence,  we  are  able 
to  obtain  estimates  of  optimal  solutions  and  optimal  values  through  estimates  of  that  distance.  In  par¬ 
ticular,  we  show  that  near-optimal  and  near-feasible  solutions  are  effectively  Lipschitz  continuous  with 
modulus  one  in  this  distance.  We  construct  a  general  class  of  approximations  of  extended  real-valued 
lower  semicontinuous  functions  that  can  be  made  arbitrarily  accurate  and  that  involve  only  a  finite 
number  of  parameters  under  additional  assumptions  on  the  underlying  metric  space. 


Keywords:  epi-convergence,  Attouch-Wets  distance,  epi-splines,  solution  stability, 

approximation  theory,  near-optimality,  near-feasibility,  rate  of  convergence. 


Date:  April  6,  2016 

1  Introduction 

Solutions  of  many  optimization  problems  are  inaccessible  by  direct  means  and  one  is  forced  to  settle  for 
solutions  of  approximate  problems.  A  central  challenge  is  then  to  ensure  that  solutions  of  approximate 
problems  are  indeed  approximate  solutions  of  the  original  problems.  Moreover,  the  degree  of  approx¬ 
imation  becomes  theoretically  and  practically  important.  The  subject  has  been  studied  extensively; 
see,  e.g.,  [17,  1,  23,  24]  for  foundations  and  [14,  25]  for  applications  in  machine  learning  and  stochastic 
optimization.  In  this  paper,  we  quantify  the  error  in  optimal  values,  optimal  solutions,  near-optimal 
solutions,  and  near-optimal  near- feasible  solutions  for  approximate  problems  defined  on  general  metric 
spaces.  In  particular,  we  obtain  a  sharp  Lipschitz-stability  result  for  near-optimal  solutions  with  a 
Lipschitz  modulus  of  1.  We  also  construct  a  class  of  “elementary”  functions  called  epi-splines  that  are 
given  by  a  finite  number  of  parameters,  but  still  approximate  to  an  arbitrary  level  of  accuracy  any  ex¬ 
tended  real-valued  lower  semicontinuous  (lsc)  functions  defined  on  a  separable  metric  space.  Since  such 
lsc  functions  abstractly  represent  a  large  class  of  optimization  problems,  epi-splines  therefore  provide 
fundamental  approximations  of  such  problems. 
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The  development  relies  heavily  on  set-convergence  of  epi-graphs,  which  goes  back  to  the  pioneering 
work  of  Wijsman  [32,  33]  and  Mosco  [22],  and  was  coined  epi-convergence  by  Wets  [31].  This  notion  of 
convergence  is  the  only  natural  choice  for  minimization  problems  as  it  guarantees  the  convergence  of 
optimal  solutions  and  optimal  values  of  approximate  problems  to  those  of  a  limiting  problem.  Quan¬ 
tification  of  the  distance  between  epi-graphs,  which  then  leads  to  a  quantification  of  epi-convergence,  is 
placed  on  a  firm  footing  in  [4,  2,  5]  with  the  development  of  the  Attouch-Wets  (aw)  distance;  see  also 
[9,  11,  12,  10].  We  follow  these  lines  and  especially  those  of  [6,  7]  that  utilize  such  quantification  as  the 
basis  for  solution  estimates  in  minimization  problems.  In  contrast  to  these  two  papers,  which  deal  with 
normed  linear  spaces,  we  consider  general  metric  spaces.  Also,  our  Lipschitz-stability  result  for  near- 
optimal  solutions  goes  beyond  that  of  [7]  as  it  does  not  require  convexity,  and  we  consider  near-optimal 
near-feasible  solutions.  We  refine  the  estimates  of  distances  between  epi-graphs  in  Mn  provided  by  [24, 
Chapter  7]  and  [29],  and  also  make  them  applicable  to  general  metric  spaces.  Approximations  of  lsc 
functions  on  Mn  by  epi-splines  is  given  by  [27].  Here,  we  extend  such  approximations  to  lsc  functions  on 
separable  metric  spaces  and  proper  metric  spaces,  and  also  give  rates  of  convergence,  which  are  novel 
even  for  Mn .  We  refer  to  [10]  for  a  general  treatment  of  topologies  on  collections  of  closed  sets;  see  also 
[1,  8,  24]  for  comprehensive  descriptions  of  epi-convergence  and  its  connections  to  variational  analysis 
broadly. 

Our  motivation  for  going  beyond  normed  linear  spaces,  which  is  the  setting  of  [5,  6,  7],  derives  from 
emerging  applications  in  nonparametric  statistics,  curve  fitting,  and  stochastic  processes  that  aim  to 
identify  an  optimal  function  according  to  some  criterion.  A  class  of  functions  over  which  such  optimal 
fitting  might  take  place  is  the  collection  of  lsc  functions  on  ]Rn,  often  simply  with  n  =  1;  see  [30,  26,  27] 
for  applications.  The  class  of  such  lsc  functions  offers  obvious  modeling  flexibility,  which  is  important 
to  practitioners,  but  under  the  aw-distance  the  class  is  a  proper  metric  space  that  fails  to  be  linear  [24, 
Theorem  7.58].  Since  it  is  proper,  every  closed  ball  in  this  metric  space  is  compact  and  the  existence  of 
solutions  of  such  optimal  fitting  problems  is  more  easily  established.  We  observe  that  the  metric  given 
to  this  class  of  lsc  functions  has  the  consequence  that  proximity  of  two  functions  implies  closeness  of 
their  minimizers.  This  property  is  often  important  in  probability  density  estimation,  where  the  focus 
is  on  the  modes  of  the  density  functions,  i.e.,  the  maximizers  of  the  density  functions.  When  fitting 
cumulative  distribution  functions,  the  metric  metrizes  weak  convergence  [29].  In  both  of  these  cases 
a  reorientation  towards  upper  semicontinuous  functions  instead  of  lsc  functions  is  needed.  In  fact, 
nearly  every  result  in  this  paper  can  be  stated  in  terms  of  extended  real-valued  upper  semincontinuous 
functions.  However,  we  maintain  the  lsc  perspective  for  simplicity. 

There  is  an  extensive  literature  on  local  stability  of  optimization  problems  under  parametric  per¬ 
turbations;  see  for  example  [13,  16,  20,  19,  21,  18,  15]  for  a  small  collection  of  references.  In  contrast  to 
these  local  stability  results,  dealing  with  “small”  perturbations  of  an  optimization  problem,  we  present 
global  results.  That  is,  we  give  estimates  of  the  distance  between  solutions  of  two  problems  that  might 
be  arbitrarily  far  apart.  The  ability  to  estimate  the  solution  of  one  problem  from  that  of  another  rather 
different  problem  is  especially  important  in  stochastic  optimization,  optimal  control,  and  semi-infinite 
programming,  and  their  numerous  applications,  as  there  we  might  only  be  able  to  construct  and  solve 
coarse  approximations  of  the  problem  of  interest. 
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The  paper  is  organized  as  follows.  After  the  review  of  epi-convergence  in  Section  2,  we  proceed  in 
Section  3  with  estimates  of  the  aw-distance.  Section  4  presents  bounds  on  solution  errors  for  optimiza¬ 
tion  problems.  Section  5  defines  epi-splines  and  discusses  their  approximation  properties. 

2  Background 

Throughout,  we  let  (A,  d)  be  a  metric  space  and  lsc-fcns(A)  :=  {/  :  X  — X  1R  :  f  lsc  and  /  ^  oo}, 
where  1R  :=  1R  U  {— oo,oo}.  Thus,  every  /  G  lsc- fens  (A)  has  a  nonempty  closed  epi-graph  epi  /  := 
{(x,  xo)  G  X  x  1R  :  /(x)  <  xo}.  For  a  nonempty  closed  B  C  X,  we  also  write  lsc-fcns(£>)  for  the  subset 
of  lsc-fcns(A)  consisting  of  functions  /  with  /(x)  =  oo  for  all  x  0  B.  When  considering  X  x  M,  we  use 
the  product  metric  d(x,  y)  :=  max{d(x,  y),  |xo  —  yo|}  for  x  =  (x,  xo)  £lxl!  and  y  =  (y,  yo)  £lx£ 
Let  IV  :=  {1,2,...}.  Convergence  is  indicated  by  —X  regardless  of  type,  with  the  being  meaning  clear 
from  the  context. 

We  recall  (see  for  example  [1,  8,  24])  that  fu:X^lR  epi-converge  to  /  :  X  — x  1R  if  and  only  if 

for  every  xv  —X  x,  liminf  fu(xu)  >  /(x),  and 
for  some  xv  — X  x,  limsup  fu(x1')  <  /(x). 

Epi-convergence  neither  implies  nor  is  implied  by  pointwise  convergence.  Uniform  convergence  en¬ 
sures  epi-convergence,  but  fails  to  handle  extended  real-valued  functions  satisfactory — a  necessity  in 
constrained  optimization  problems. 

For  /  :  X  —X  R,  C  C  X,  and  £  >  0,  let  inf  /  :=  inf{/(x)  :  x  G  X},  infc  /  :=  inf{/(x)  :  x  G  C}, 
argrnin  /  :=  {x  G  A  :  /(x)  =  inf  /},  and  £-  argmiri  /  :=  {x  G  A  :  /(x)  <  inf  /  +  e}.  It  is  well  known 
that  epi-convergence  ensures  convergence  of  solutions  of  minimization  problems  (see  for  example  [24, 
Chapter  7]  and  [3,  1]): 

2.1  Proposition  (convergence  of  minimizers)  Suppose  that  fv  :  A  — x  M  epi-converges  to  f  :  X  — x  M. 
Then, 

limsup  (inf  fu)  <  inf  /. 

Moreover,  if  xk  G  argmiri /'/fc  and  xk  x  for  some  increasing  subsequence  {ui,  U2, ...}  C  IV,  then 
x  G  argrnin  /  and  lim^oo  inf  fVk  =  inf  /. 

A  strengthening  of  epi-convergence  ensures  the  convergence  of  minima  and  approximation  of  mini¬ 
mizers  (see  for  example  [28]). 

2.2  Definition  (tight  epi-convergence)  The  functions  fu:X^M  epi-converge  tightly  to  f  :  X  — x  1R 
if  they  epi-converge  to  f  and  for  all  e  >  0,  there  exists  a  compact  set  Ke  C  A  and  an  integer  vE  such 
that 

inf  xe  fv  <  inf  fv  +  £  for  all  v  >  ve . 

2.3  Proposition  (convergence  of  infima)  Suppose  that  fu  :  X  — x  fR  epi-converges  to  f  :  X  — x  fR  and 
inf  /  is  finite.  Then,  fv  epi-converges  tightly  to  f 
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(i)  if  and  only  if  inf  fv  inf  f; 

(ii)  if  and  only  if  there  exists  a  sequence  eL'\0  such  that  ev -  argmiri  fv  set-converges1  to  argmin  /. 

Throughout,  let  x  €  X  be  fixed.  The  choice  of  x  can  be  made  arbitrarily,  but  results  might  be  sharper 
if  x  is  somewhat  near  minimizers  of  functions  of  interest  as  the  analysis  relies  on  the  intersection  of 
epi-graphs  with  Sp  :=  fBp  x  [—p,p\,  where  JBp  :=  lB(x,p)  :=  {x  €  X  :  d(x,x)  <  p}  and  p  >  0.  The 
Attouch-Wets  (aw)  distance  d  (given  x)  is  defined  for  any  f,g  G  lsc-fcns(X)  as 

POO 

d(f,g)  ■=  /  dP(f,g)e~pdp, 

J  o 

where  the  p-aw-distance ,  p  >  0,  is  given  by 

dp(f,g)  ■=  sup  { I dist  (x,epi/)  -  dist  (z,epicj)]  :  x  €  Sp}  , 

with  dist  giving  the  usual  point-to-set  distance,  which  here  is  given  by 

dist  ( x ,  C )  =  inf  {d(x,  y)  :  y  €  (7}  if  C  C  X  x  1R  is  nonempty  and  dist(x,  0)  =  oo. 

This  setup  resembles  that  of  [24,  Section  7.1],  but  there  X  =  lRn  and  the  Euclidean  distance  is  used  on 
lRn  x  M.  Generally,  (lsc-fcns(X),  d)  is  a  metric  space  [10,  Section  3.1]  that  is  complete  whenever  ( X ,  d) 
is  complete  [10,  Theorem  3.1.3].  We  conclude  from  [10,  Theorem  3.1.7]  that  for  /",/  €  lsc-fcns(X), 

d(fu,f)  — >•  0  implies  that  fv  epi-converges  to  /. 

We  recall  that  a  metric  space  is  proper  if  every  closed  ball  in  that  space  is  compact.  If  (X,  d)  is  proper, 
then  the  converse  also  holds:  fv  epi-converges  to  /  implies  that  d(fu ,  /)  — >•  0  (see  [10,  Theorem  3.1.7] 
and  [5,  Theorem  4.2,  Lemma  4.3],  with  the  latter  results  being  stated  for  Mn  but  their  proofs  carry  over 
to  the  case  of  proper  metric  spaces).  In  addition  to  X  =  Mn  with  the  usual  metric,  (X.  d)  is  proper 
when  X  =  lsc-fcns(Jin)  and  d  =  d.  In  fact,  (lsc-fcns (lRn),d)  is  a  proper  complete  separable  metric 
space;  [24,  Theorem  7.58]  and  [27,  Corollary  3.6].  This  example  is  a  motivation  for  the  development 
due  to  applications  in  nonparametric  statistics,  curve  fitting,  and  stochastic  processes;  see  [26,  27,  30]. 
We  use  the  following  well-known  fact  repeatedly. 

2.4  Lemma  If  (' Y ,  dy)  is  a  proper  metric  space,  then  argminjdy  (y,  y')  :  y'  €  C}  /  0  for  every  y  G  Y 
and  nonempty  closed  set  C  C  Y . 

Since  the  aw-distance  quantifies  epi-convergence,  it  is  clear  that  its  value  for  two  lsc  functions,  or  that 
of  its  estimates,  leads  to  bounds  on  the  distance  between  optimal  solutions  and  optimal  values  of  the 
two  functions.  Estimates  of  the  aw-distance  is  the  subject  of  the  next  section,  with  solutions  being 
dealt  with  in  Section  4. 

1The  outer  limit  of  a  sequence  of  sets  {Wjvgjv,  denoted  by  limsup A" .  is  the  collection  of  points  x  to  which  a 
subsequence  of  {x1,}^£n,  with  x1'  £  Av ,  converges.  The  inner  limit,  denote  by  liminf  A",  is  the  points  to  which  a  sequence 
of  {xv}„zn,  with  xv  £  A1' ,  converges.  If  both  limits  exist  and  are  identical  to  A,  we  say  that  Av  set-converges  to  A;  see 
[10,  24], 
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3  Distance  Estimates 


This  section  gives  practically  important  estimates  of  the  aw-distance  between  two  lsc  functions.  We 
begin  with  defining  an  auxiliary  quantity  that  estimates  dp.  For  p  >  0  and  /, g  G  lsc-fcns(A),  let 

dp(f,g)  ■=  max  |e(epi/n§p,  epic/),  e(  epig  n  Sp,  epi  /)  j, 

where  the  excess  of  a  set  C  over  a  set  D  is  given  by 

e(C,D )  :  =  sup{dist(z,  D)  :  z  €  C}  if  C,  D  are  nonempty, 

e(C,D)  =  oo  if  C  nonempty  and  D  empty,  and  e(C,D)  =  0  otherwise.  Roughly  speaking,  dp(f,g)  is 
the  “padding”  of  epi  g  needed  for  it  to  contain  epi  /  fl  §p  and  vice  versa.  The  relations  among  d,  dp, 
and  dp  given  next  extend  [24,  Exercise  7.60]  from  X  =  Mn  to  metric  spaces. 

3.1  Proposition  (estimates  of  aw-distance)  For  f,g£  lsc-fcns(X),  the  following  holds,  where  we  use 
the  notation  8j  =  dist((x,  0),  epi  /)  and  similarly  for  g. 

(i)  dp(f,g)  and  dp(f,g )  are  nondecreasing  functions  of  p; 

(ii)  dp'(f,g)  -  dp(f,g )  <  2e(Sp/,§p)  for  p'  >  p  >  0; 

(Hi)  dp(f ,  g)  <  dp(f,  g)  <  dp>(f ,  g)  for  p'  >  2p  +  ma x{5f,  for  p  >  0; 

(iv)  dp(f ,  g)  <  ma x{5f,  <5P}  +  p  for  p  >  0; 

(v)  d(f,g )  >  (1  -  e~p)\ Sf  -  Sg\  +e~pdp(f,g )  for  p  >  0; 

(vi)  d(f,  g)  <  (1  -  e~p)dp{f ,  g)  +  e~p[max{6fl  5g}  +  p  +  1]  for  p  >  0; 

(vii)  | Sf  -  5g\  <  d(f,g)  <  max{8f,8g}  +  1. 

If  ( X ,  d)  is  proper,  then  >  can  he  replaced  by  >  in  (iii). 

Proof.  See  appendix.  □ 

A  nearly  precise  estimate  of  dp  is  provided  by  the  following  convenient  quantity,  which  is  closely 
related  to  the  Kenmochi  condition  of  [5].  For  /, g  G  lsc-fcns(X),  p  >  0,  and  8  >  0,  let 

dp(f,  g)  ■=  inf  1 77  >  0  :  infB(xfl+<5)  g  <  max{/(x),  -p)  +  p,Mx  €  levp  /  n  Bp 

iniB(x,r,+S)  f  <  max{g(x),  -p}  +  qXx  G  levp£f  n  Bp  |, 

where  levQ  /  :=  {x  G  X  :  f(x)  <  a}.  Below,  we  also  let  dom  /  :=  {x  G  X  :  f(x)  <  00}.  The  next 
proposition  extends  a  result  in  [29]  from  X  =  Mn  to  general  metric  spaces. 
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3.2  Proposition  (estimates  for  auxiliary  quantity)  For  /,  g  G  lsc-fcns(X),  p  G  [0,  oo),  and  5  >  0, 


dSp(f,9 )  <  dp(f,g )  <  d°p(f,g)  <  oo. 

If  (X,  d )  is  proper,  then  d°p(f,  g)  =  dp(f,  g). 

Proof.  See  appendix.  □ 

We  state  next  an  upper  bound  on  dp  in  the  case  of  Lipschitz  continuous  functions.  We  say  that  a 
function  /  :  X  — »•  1R  is  Lipschitz  continuous  with  modulus  k  :  [0,  oo)  — >•  [0,  oo)  (relative  to  x)  if 

I  f(x)  -  f(y)  |  <  n(p)d(x,  y)  for  all  x,y  G  Bp  and  p  >  0. 

Parallel  to  dp(f,g),  we  also  define  with  a  slight  abuse  of  notation 

dp(C ,  D)  :=  max  j e(C  n  lBp,  D) ,  e(D  n  C)  |,  p  >  0,  C,  D  C  X  nonempty  closed. 

The  next  result  generalizes  a  statement  in  [29]  to  metric  spaces  and  also  tightens  it  slightly.  We  define 
for  any  C  C  X  the  function  lc  ■  X  1R  that  has  ic{x)  =  0  if  x  G  C  and  ic{x)  =  oo  otherwise.  We 
also  adopt  the  usual  convention  that  — oo  +  oo  =  oo. 

3.3  Proposition  (distance  estimate  for  Lipschitz  functions)  Suppose  that  C,  D  C  X  are  nonempty 
closed  sets  and  f,g:X^-M  are  Lipschitz  continuous  with  common  modulus  k.  Then,  for  p  £  [0,  oo) 
and  p'  G  (p  +  dp(C ,  D ),  oo), 

d°PU  +  1*0,9  + ld)  <  supAp  |/  -s|  +  max{l,  n(p')}cip(C,  D) 

where  Ap  =  (1  evp{/  +  iC}  U  levp{5  +  tB})  n  Bp. 

If  ( X ,  d)  is  proper,  then  the  result  also  holds  for  p'  =  p  +  dp(C,  D). 

Proof.  See  appendix.  □ 

Sometimes  the  following  asymmetric  quantity  is  useful.  For  f,g£  lsc-f'cns(X)  and  p  >  0,  let 

Vp(f',g)  ■=  inf  {??  >  0  :  infB(Xj^)  /  <  ma x{g(x),-p}  +  t?  ViG  levpgnlBp}. 

Obviously,  dp{f,g )  =  max{ri+(f-,g),r]+(g-,f)}  and  7/+(/;y)  <  max{0,sup \eVf>gnBp  f  ~  g}-  Proposition 
3.2  implies  that  for  any  /. g  G  lsc-fcns(W),  r}p(f’,g)  <  oo. 

4  Solution  Estimates 

Since  f,g£  lsc-fcns(W)  completely  dehne  the  problems  minxe_v  f(x)  and  min x£x  g(x),  and  d  quantifies 
epi-convergence,  it  is  clear  that  d(f,g )  leads  to  estimates  of  |  inf  /  —  inf g \  as  well  as  some  notion 
of  distance  between  argmin  /  and  argiriin  g.  In  this  section,  we  provide  such  estimates  as  well  as 
estimates  between  near-optimal  solutions  and  near-optimal  near-feasible  solutions.  Instead  of  d,  we 
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work  directly  with  the  auxiliary  quantity  dp,  which  is  simpler  to  estimate  in  most  practical  situations; 
see  for  example  Proposition  3.3.  In  view  of  Propositions  3.1  and  3.2,  the  difference  between  the  two 
quantities  are  anyhow  small  for  large  p.  We  note  that  the  results  of  this  section  are  practically  more 
useful  when  function  values  are  scaled  to  be  of  the  same  order  of  magnitude  as  the  parameter  p. 

We  start  by  developing  a  result  for  optimal  values  that  generalizes  a  statement  in  [6]  by  considering 
general  metric  spaces,  permitting  empty  sets  of  optimal  solutions,  and  dealing  with  the  asymmetric 
quantity  77+. 

4.1  Theorem  (approximation  of  optimal  value)  Suppose  that  f,g  £  lsc-fcns(X),  p  £  (0, 00),  p  > 
inf  /  >  —p,  and  e-  argmin  /  n  JBp  7^  0  for  all  e  >  0.  Then, 

inf  g  -  inf  /  <  77+  (5;  /)  <  d°p(f,  g). 

If  the  assumption  about  f  also  holds  for  g,  then 

I  inf  g  —  inf  f\  <  d%(f,g). 

Proof.  We  note  that  77+  (<7;  /)  <  00.  Let  77  £  (77+  (77;  /) ,  00)  be  arbitrary.  Then,  for  all  x  £  levp  /  D  lBp, 

inf  g  <  infB(Xfl)  g  <  max{/(i),  -p]  +  77.  (1) 

Set  £q  =  p  —  inf  /  >  0.  Let  e  £  (0,£o]  be  arbitrary  and  x£  £  e-  argmin  fn!Bp.  Then,  f(xe)  <  inf  f  +  e  < 
inf  /  +  £q  =  p  and  thus  xe  €  levp  /.  Applying  (1)  with  x  =  x£  results  in 

inf  g  <  max{/(x£),  —  p}  +  77  <  maxjinf  /  +  e,  —  p}  +  77  <  inf  /  +  e  +  77. 

After  letting  e  and  77  tend  to  their  lower  limits,  we  obtain  that  inf  77  <  inf  f  +  g^{g~,  f )  <  inf  f  +  dp(f,g). 
The  final  result  follows  after  a  replication  of  these  arguments  with  the  roles  of  /  and  g  reversed.  □ 

We  observe  that  if  argmin  /  n  H3p  ^  0,  then  it  suffices  to  have  p  >  inf  /  >  —p  in  Theorem  4.1. 

To  enable  a  statement  about  optimal  solutions,  we  need  to  bring  in  conditioning.  The  next  result, 
which  generalizes  a  similar  statement  in  [6]  to  metric  spaces,  carries  this  out. 


4.2  Theorem  (approximation  of  optimal  solutions)  Suppose  that  f \g  €  lsc-fcns(X)  are  such  that 

inf  /,  inf  g  £  [—p,  p],  argmin  /  n  lBp  /  0,  and  argmin  g  n  lBp  /  0 

for  some  p  £  [0,  00)  and  that  there  exists  an  increasing  and  continuous  function  ipf  :  [0,oo)  — >•  [0,  00), 
with  ipf( 0)  =  0  such  that 

f(x)  —  inf  />/’/(  dist(x,  argmin  /))  for  all  x  £  X. 


Then, 


e  (argmin g  n  Bp,  argmin  /)  <  d°p(f,  g)  +  1  ^2 d°p(f,  gfj  . 
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Proof.  From  Proposition  3.2,  dp(f,  g)  <  oo.  Let  g  G  (dp(f,  g),  oo)  be  arbitrary.  For  all  x  €  levp  gH]Bp, 
inf/3(x,r/)  /  <  max{y(x),  —  p}  +  g-  In  view  of  the  property  of  '(/’/  and  the  fact  that  inf  g  >  — p,  we  find 
that  for  x  €  levpy  n  S3p, 

g{x)  -  inf  /  +  g  =  max{y(x),  -p}  +  7/  -  inf  /  >  infyeB(a.)J?)  /(y)  -  inf  / 

>  infyeB(x,v)  V’/  ( dist(y,  argmin  /)) . 

By  Theorem  4.1,  inf  g  —  inf  /  <  77;  see  the  comment  after  that  theorem  to  establish  that  p  is  sufficently 
large  despite  the  fact  that  it  might  coincide  with  inf  /  and  inf  g.  Thus,  for  x  G  argminy  n -2?^,  which  of 
course  implies  that  x  G  levpy,  we  have  that 

2g  >  inf  g  —  inf  /  +  g  =  y(x)  —  inf  f  +  g 

>  inf yeB(x,v)  Tpf  ( dist(y,  argmin  /)) 

>  V’/  ( inf yeB{x,v)  dist(y,  argmin  /)) , 

where  the  last  inequality  follows  from  the  increasing  property  of  V’/-  Therefore,  we  have  that 

inf yeB(x,v)  dist  (y,  argmin/)  < 

Let  s  >  0  be  arbitrary.  There  exists  an  x£  G  ]B{x,rj)  such  that 

dist  (xe,  argmin  /)  <  infyeB(x^  dist  (y,  argmin  /)  +  e. 

These  facts  then  imply  that  for  x  G  argmin  g  n  JBp , 

dist  (a;,  argmin  /)  <  dist(x£,  argmin  /)  +  d(x£,  x)  <  inf  y&B(x^)  dist  ( y ,  argmin  /)  +  e  +  r\ 

<  'ipj1(2r])  +  e  +  rj. 

Since  e  >  0  is  arbitrary,  we  have  that  dist(x,  argmin/)  <  rj  +  ipJ  1(2y).  This  holds  for  all  x  G  argmin g n 
1BP  and  consequently 

e  ( argmin  g  n  IB p,  argmin  /)  <  g  +  ^J1(2g). 

Since  'ipj1  is  continuous,  the  conclusion  follows  by  letting  r/  tend  to  d°p(f,g).  □ 

The  bound  is  sharpe  even  for  X  =  ]R  as  demonstrated  by  the  following  example.  Let  f(x )  =  x 2 
and  for  rj  >  0  let  gv(x)  =  (x  —  g)2  if  x  G  JR  \  ([0,  2rj\  U  {g  +  y/2g}),  gv(x)  =  g  if  x  G  [0,  2g\  U  {g  +  \Z2g}- 
One  can  show  that  for  x  =  0  and  sufficiently  large  p,  dp(f,g)  =  g.  Then,  argmin/  =  {0}  and 
argmin  gr/  =  [0, 2g\  U  {g  +  \/2p}-  Since  the  conditioning  function  =  t2  in  this  case,  we  see  that  the 
conclusion  of  the  theorem  holds  with  equality  when  g  <  2. 

In  practice,  it  is  difficult  to  develop  a  conditioning  function  ifrf  as  required  by  Theorem  4.2;  see 
[6]  for  a  thorough  discussion.  Fortunately,  a  strong  Lipschitz-type  statement  can  be  made  about  near- 
optimal  solutions  without  the  knowledge  about  such  conditioning.  Even  for  X  =  Mn ,  the  next  result  is 
novel  by  considering  different  levels  of  near  optimality  for  the  two  problems  and  avoiding  the  convexity 
assumption  of  [7]  and  [24,  Theorem  7.69]. 


4.3  Theorem  (approximation  of  near-optimal  solutions)  Suppose  that  f,  g  E  lsc-fcns(X),  p  E  (0,  oo), 
inf  /  €  [—p,  p) ,  and  7-  argmin  /  n  1BP  /  0  for  all  7  >  0.  If  £  >  0,  6  >  0,  and  inf  g  E  [— p,  p  —  s\ ,  then 

e (e- argmiri g  n  1BP,  (e  +  2fj  +  <5)-  argmin  /)  <  fj,  where  fj  =  dp(f,g). 

If  in  addition  (X,  d )  is  proper,  then  6  can  be  set  to  zero. 

Proof.  From  Proposition  3.2,  dp(f,  g)  <  00.  Let  7  E  ( dp(f ,  g),  00)  be  arbitrary.  For  all  x  E  levp  gfMBp , 
,v)  f  <  max{g(x),  — p}  +  77.  Let  x  E  e-  argmin  g  n  lBp,  which  implies  that  x  E  levpp.  Since 
inf  g  >  — p,  we  therefore  have  that 

inftf(x  ,r?)  /  <  max{p(x),  — p}  +  f]  <  maxjinf  g  +  e,  — p}  +  77  =  inf  g  +  e  +  77.  (2) 

There  exists  x«s  E  lB(x,  rj)  such  that  f{x§)  <  iniB/x  ^  f  +  6/2.  From  above  we  then  have  that  f(x$)  < 
inf  g  +  e  +  rj  +  5/2  <  inf  f  +  e  +  2r]  +  6/2,  where  the  last  inequality  follows  from  Theorem  4.1.  Thus, 
Xfj  E  (e  +  2?7  +  6/2)-  argmin  /  and  dist(x,  (s  +  277  +  6/2)-  argmin  /)  <  77.  Since  this  holds  for  all 
x  E  £-  argmin  g  n  IB  p, 

e(e-  argmin  g  n  lBp,  (e  +  2g  +  6/2)-  argmin  f)  <  77. 

The  first  conclusion  then  follows  after  letting  77  tend  to  77. 

If  ( X ,  d)  is  proper,  then  we  continue  from  (2)  as  follows.  Since  lsc  functions  attains  their  minimum 
over  compact  sets,  there  exists  x  E  argmin^^  ^j  /  and  thus  f(x)  <  inf  g  +  e  +  77  <  inf  f  +  e  +  2g, 
where  again  the  last  inequality  follows  from  Theorem  4.1.  Consequently,  x  E  (s  +  277)- argmin  /  and 
dist(x,  (e+277)-  argmin/)  <  77.  Let  {r]u}u£^  be  such  that  7]u\g  and  dist(x,  {£+2rf)-  argmin  /)  <  77^.  In 
view  of  Lemma  2.4,  there  exists  yv  E  (e+277^)-  argmin  /  such  that  d(x,  yv)  =  dist(x,  (e+277^)-  argmin  /). 
Since  {yu}ueN  is  contained  in  a  ball,  which  under  the  additional  assumption  is  compact,  we  have  that 
there  exists  an  N  C  IN  and  y  such  that  yv  — > y.  Since  d(x ,  yu)  <  for  all  u,  d(x,  y)  <  fj.  Moreover, 
f(yu)  <  e+2r]u  for  all  u  implies  that  f(y)  <  £+21)  because  /  is  lsc.  Thus,  dist(x,  (e +2fj)-  argmin/)  <  fj. 
Since  this  holds  for  all  x  E  e-  argmin  g  fl  lBp,  the  second  conclusion  follows.  □ 

The  above  bound  is  sharp  even  for  X  =  1R.  Suppose  that  f,g\M^IR  are  given  by  f(x)  =  1  for 
x  E  [1,2),  /( 2)  =  —1,  and  f(x)  =  00  otherwise,  and  g{x)  =  0  for  x  E  [0,2]  and  g(x)  =  00  otherwise. 
Obviously,  f,g  are  lsc.  Let  dp  be  defined  with  x  =  0  and  p  >  2.  Then,  dp(f,g)  =  1.  Clearly,  x  =  0 
is  in  argmin g  <lIBp  and  dist(x,  2-  argmin  /)  =  1.  In  fact,  e(argmin g  n  lBp,  2-  argmin  /)  =  1.  Moreover, 
dist(x, 7- argmin /)  =  2  for  7  E  [0,2). 

Theorem  4.3  leads  to  the  following  corollary  about  rate  of  convergence. 

4.4  Corollary  (rate  of  convergence  to  near-optimal  solutions)  Suppose  that  p  E  (0,  00),  6  >  £  >  0, 
/  E  lsc-fcns(X),  inf  /  E  [—p,p),  and  7- argmin  /  n  lBp  /  0  for  all  7  >  0.  If  fu  E  lsc-fcns(X)  has 
inf  fv  E  [—p,  p  —  e]  for  all  v  and  dp(f ",  /)  —f  0,  then  there  exists  an  u  such  that 

e (e- argmin  fu  fl  lBp,  6- argmin/)  <  d^(fu,  f)  for  all  v  >  v. 
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Proof.  By  Theorem  4.3  we  have  that  for  any  <5o  >  0  and  u, 

e(e-  argmin  fv  n  Bp ,  (e  +  2 rf  +  <5q)-  argmin  /)  <  rf ,  where  if  =  dp(/1' ,  /). 

Set  <5o  =  {8  —  e)/ 2  >  0.  Since  77^  -»  0,  there  exists  an  P  such  that  rf  <  (8  —  e)/4  for  all  v  >  u.  Since 
e  +  2viv  +  8q  <  e  +  2((5  — e)/4  +  ((5  — e)/2  =  5  for  such  u.  we  have  that  e (e-  argmin  fuDBp,  8-  argmin/)  < 
77^  for  v  >v,  which  establishes  the  conclusion.  □ 

Near-optimal  solutions  are  feasible  in  the  sense  that  x  €  e-  argmin/  and  inf  /  <  oo  implies  that 
x  £  dom  /.  We  also  consider  near- feasibility,  which  is  often  practically  relevant,  and  reach  the  following 
results  about  level  sets;  see  [7]  for  results  about  convex  functions  on  norrned  linear  spaces. 

4.5  Theorem  (approximation  of  level  sets)  Suppose  that  f,  g  £  lsc-fcns(X),  p  £  [0,  oo),  and  8  £  [— p,  p\. 
Then,  for  any  7  >  0, 

e(lev«5  g  n  Bp,  lev<5_|_jj+7  /)  <  fj  =  77+ (/;  g ) 

If  in  addition  ( X ,  d)  is  proper,  then  7  can  he  set  to  zero. 

Proof.  We  observe  that  77+  (/;  g)  <  00.  Let  rj  £  (?/+(/;</),  00)  be  arbitrary.  For  all  x  £  levpgn  Bp, 
miB(x,r})  f  <  nrax{<7(x),  —  p}  +  r/.  Let  x  €  leva  g  n  Bp,  which  implies  that  x  £  levp^.  Thus, 

f  <  ma x{g(x),  -p}  +  rj  <  max{<5,  -p]  +  rj  =  5  +  rj.  (3) 

There  exists  x1  £  B(x,  rf)  such  that  /(x7)  <  inf^^  ^)  /  +  7/2.  From  above  we  then  have  that  f{xfj  < 
8  +  77  +  7/2.  Consequently,  a:7  £  leva+r?+7/2  /  and  dist(x,  leva+p+7/2  /)  <  ??•  Since  this  holds  for  any 
.T  £  leva  g  FI  Bp,  e(  leva  g  n  Bp,  1  eva+J?+7;/2  /)  <  rj.  The  first  conclusion  then  follows  after  letting  77  tend 
to  fj. 

If  (X,d)  is  proper,  then  we  continue  from  (3)  as  follows.  Since  /  attain  its  minimum  over  B(x,r f 
in  this  case,  there  exists  x  £  argiriinR(X;?;j  /  and  f(x)  =  inf#^)  /  <  8  +  77.  Thus,  x  £  1  eva+p  /  and 
dist(x,  leva+p /)  <  77.  Let  be  such  that  77 v\fj  and  dist(x,  leva+^  /)  <  gv .  In  view  of  Lemma 

2.4,  there  exists  yv  £  leva+r^  /  such  that  d(x,yu)  =  dist(x,  leva+p^  /).  Since  {yl'}v^N  is  contained  in 
a  ball,  which  under  the  additional  assumption  is  compact,  we  have  that  there  exists  an  JV  C  IV  and  y 
such  that  'if  ~^N  y.  Since  d(x,  yu)  <  for  all  v,  d(x,  y)  <  fj.  Moreover,  f(yu)  <  8  +  rf  for  all  v  implies 
that  f(y)  <  8  +  fj  because  /  is  lsc.  Thus,  dist(x,  le\7+p  /)  <  fj.  Since  this  holds  for  any  x  €  lev^  g  n  Bp, 
the  second  conclusion  follows.  □ 

When  considering  both  near-optimality  and  near-feasibility,  we  adopt  the  following  definition.  For 
£,  8  >  0,  the  set  of  near-optimal  near-feasible  solutions  of  the  problem  min{/o(x)  :  f(x)  <  0,x  €  X}, 
which  of  course  is  equivalent  to2  min{/o  +  if<o},  is  given  by 

{£,  8)-  argmin {/o  +  i/<0}  :=  {x  £  X  :  f0(x)  <  inf {/0  +  tf<0}  +  e,  f(x)  <  <5}. 

The  next  results  are  the  first  ones  dealing  with  near-optimality,  near-feasibility,  and  the  asymmetric 
quantity  77+  in  a  general  setting. 

2We  use  the  slight  abbreviation  t/<0  for  C{xex-.f(x)<o}- 
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4.6  Theorem  (approximation  of  near-optimal  near-feasible  solutions)  For  e,p,  77,770  G  [0,  oo),  suppose 
that  the  functions  fo,go,  f,g  €  lsc-fcns(X)  satisfy 

0)  supBp{g0  -  foj  <  Vo  and  sup Bp{g  -  f}  <  y 

(H)  inf{/o  +  L  f<-V}  hnite  and  inf{g0  +  %<o}  €  [-p,  p-e] 

(in)  argmin{/0  +  if<-v}  n  Bp^H)  and  argmin{g0  +  ifl<o}  n  Bp  /  0. 

Then,  for  a,  6,  >  0  and  7  >  0, 

e((s,  <5)-argmin{<7o  +  t9<0}  n  Bp,  (e  +  77+  +  ??0  +  7,  a)-argmin{/0  +  <-/<_,,})  <  g+ , 

where  77+  =  77+ (/0  +  Lf<a-V;  5o  +  h?<s)  <  00. 

If  in  addition  ( X ,  d )  is  proper,  then  7  can  be  set  to  zero. 

Proof.  First  we  note  that  77+  (/o  +  i-j<a-rf  go  +  ^<<5)  is  finite  because  /o  +  t/<Q_T 7  and  go  +  %<<5  are  in 
lsc-fcns(X).  Second,  let  77  G  (?7+,oo).  Then, 

infB(i,»j){/o  +  t/<Q-r?}  <  max{g0(x)  +  t9<5(x),  -p}  +  fj  for  all  x  €  levp{g0  +  ^<5}  H  Bp. 

For  x  €  Bp  satisfying  go(x)  <  inf{go  +  <9<o}  +  £  and  g(x)  <  5,  we  have  x  G  levp{go  +  ^<<5} -  Since 
inf  (go  +  ig<  0}  >  ~P, 

inf_B(a;,jj){/o  +  t/<a-r,}  <  max{g0(x)  +  t9<5(x),  -p}  +  fj  <  max{g0(x),  -p}  +  g 

<  inf {g0  +  ig< 0}  +  e  +  fj  =  inf Bp{go  +  ig<o}  +  e  +  fj  <  00,  (4) 

where  the  equality  follows  from  the  fact  that  argminjgo  +  ig<o)  H  Bp  /  0.  We  next  consider  two  cases. 
First,  if  inlB(x,rj){fo  +  Lf<a-r /}  is  finite,  then  there  exists  y  G  B(x,g)  such  that 

fo(y)  +  lf<a-r,(y)  <  infB(x^){/o  +  +  7/2- 

Consequently, 


/o(y)  +  l'f<a-rt(y)  <  inf_Bp{go  +  tg< o}  +  £  +  fj  +  7/2 

<  inf_Bp{/o  +  t9<0}  +  e  +  fj  +  ??0  +  7/2 

<  inf_Bp{/o  +  <-/<-»?}  +  e  +  fj  +  go  +  7/2, 

where  the  last  inequality  follows  from  the  fact  that  f(z)  <  —g  implies  that  g(z)  <  0  for  z  G  Bp.  Since 
the  right-hand  side  above  is  finite,  f(y)  <  a  —  g  and  fo(y)  <  inf#  {/o  +  t/<-r/}  +  e  +  ??  +  ho  +  7/2- 
Since  inf#  {/o  +  Lf<-r /}  =  inf{/o  +  i/<-r?};  we  then  have  that 

y  G  (e  +  fj  +  ?7q  +  7/2,  a)-  argmin{/0  + 


Hence, 

dist  (x,  (e  +  i)  +  %  +  7/2>  «)-  argmin{/0  +  t/<-9})  <  fj-  (5) 
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Second,  we  consider  the  other  case  when  miB(x,ri){fo  +  Lf<a-t)}  =  —  oo.  Then,  there  exists  y  £  Z?(x,  77) 
such  that  again  f(y)  <  a  —  77  and  fo(y)  <  inf Bp{fo  +  Lf<-g}  +  £  +  77  +  77 o  +  7/2.  Thus,  also  in  this  case, 
(5)  holds.  Since  this  argument  is  valid  for  all  x  £  Bp  satisfying  go(x)  <  inf{<7o  +  ts<o} +  £  and  g(x )  <  5, 

e((e,  5)-  argrnin{ry0  +  tg<0}  n  Bp,  (e  +  77  +  r/0  +  7/2,  a)-  argmin{/0  +  t/<-r?})  <  V- 

The  main  conclusion  then  follows  after  letting  77  tend  to  its  lower  limit  of  y+ . 

If  (X,  d )  is  proper,  then  we  continue  from  (4)  by  recognizing  that  there  exists  y  £  arginin^ pp{fo  + 
Lf<a-rj}  and  thus 

fo(y)  +  tf<a-v(,y)  <  inffipiffo  +  L?< 0}  +  £  +  fj  <  infBp{/o  +  if<-v}  +  fj  +  V 0  +  £  <  00. 

Consequently,  f(y)  <  a  -  77  and  /0(y)  <  infBp{/0  +  +  fj  +  ??0  +  £,  or  equivalently,  y  €  (e  + 

77  +  770,  a)- argmin{/o  +  Following  the  same  argument  as  used  towards  the  end  of  the  proof  of 

Theorem  4.3,  we  find  that  the  relation  also  holds  as  fj  reaches  its  lower  limit  of  77 +  and 

dist  (x,  (e  +  77 +  +  770,  a)-  argmin{/0  +  if<-v})  <  V+ ■ 

Since  this  holds  for  all  x  £  1BP  with  go(x)  <  inf {50  +  ig< 0}  +  £  and  g(x)  <  6,  the  conclusion  follows. □ 
We  next  give  an  estimate  of  77^  analogous  to  Proposition  3.3. 

4.7  Proposition  (bounds  for  Lipschitz  continuous  objective)  Suppose  that  f,fo,g,go  €  lsc-fcns(X), 
with  fo  being  Lipschitz  continuous  functions  with  modulus  k,  and  inf{/o  +  if< 0}  is  finite.  Then,  for 
a,  5,  p  £  [0, 00)  and  p'  £  (p  +  e(lev,s  g  (7  lBp ,  leva  /),  00). 

Vp  ( fo  +  9o  +  tg<s)  <  nrax{0,  supAo  /0  -  go}  +  max{l,  n(p')}e( levs  g  (7  Bp,  levQ  /), 

where  A°p  =  1  evp{g0  +  ig<s}  (7  Bp. 

Moreover,  if  in  addition  5  <  p,  and  a  >  77 ■+(/;  g)  +  6,  then 

e0ev<5  g  (7  Bp,  levQ  /)  <  77+  (/;  g). 

If  ( X ,  d)  is  proper,  a  =  ??+(/;  g)  +  6  is  permitted. 

Proof.  We  observe  that  r/p(fo  +  t/<a;  go  +  t-g<s)  is  finite  even  if  go  +  tg<s  is  identical  to  00  and 
that  1  eva  /  /  0.  Thus,  e(lev^  /  (7  Bp,  leva  /)  <  00.  Let  p'  £  (p  +  e(lev,5  g  (7  Bp,  leva  /),  00),  and 
£  £  (0,  p'  -  p  -  e(lev5  g  (7  Bp,  lev,,  /)].  Set 

Ve  =  nrax{0,  sup^o  fo  ~  go}  +  nrax{l,  n(p')}  [e(lev5  g  (7  Bp,  leva  /)  +  s\ . 

Suppose  that  x  £  levp{(7o  +  ig<$}  17  Bp,  which  implies  that  g(x)  <  5.  Since  lev^  /  /  0,  there  exists 
y  £  X  such  that  f(y)  <  a  and  d(x,y )  <  inf {d(x,y')  :  f{y')  <  a,y'  €  X}  +  e.  Thus, 

e(lev5  g  (7  Bp,  lev,,  /)  >  dist(x,  leva  /)  >  d(x,  y)  -  e. 
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Consequently,  d(x,y )  <  g£  and  d(x,y)  <  d(x,x)+d(x,y)  <  /9+e(lev,5  gfMBp,  levQ  /)+£  <  p' .  Combining 
these  facts,  we  find  that 

+  if<a}  <  My)  =  My)  -  Mx)  +  Mx)  -  Mx)  +  go(x ) 

<  n(p')d(x,  y)  +  max{0,  supAo  /o  -  7o}  +  max-fyo^)  +  ig<s(x),  ~p} 

<  Ve  +  max{<7o(x)  +  tg<s(x),  -p}. 

Thus,  ?7+(/o  +  t-f<a',9o  +  Lg<s)  <  he-  Since  this  fact  holds  for  all  e  >  0,  the  first  conclusion  follows. 

An  application  of  Theorem  4.5  results  in  the  second  part  of  the  result.  □ 

We  obtain  a  rate  of  convergence  result  by  combining  Theorems  4.5,  4.6,  and  Proposition  4.7. 


4.8  Proposition  (rate  of  convergence  to  near-optimal  near-feasible  solutions)  For  (3>e>0,  a>5> 
0,  p'  >  p  >  5,  and  76  (0,  a  —  S),  suppose  that  the  functions  /o,  /q  ,  /,  fv  €  lsc-fcns(X) ,  with  fo  being 
Lipschitz  continuous  with  modulus  k,  satisfy 


(i)  inf{/0  +  tf< 0},  inf{/0  +  </<-7}  are  finite 

(ii)  argmin{/0  +  i/<_7}  C  Bp  /  0 

(Hi)  inf {/((  +  if*< 0}  €  [-p,  p-e)  for  all  v 
(iv)  argmin{/Q  +  tff< 0}  H  lBp  /  0  for  all  u. 

If  supBp  |/q  —  /o|  — >•  0  and  supBp  \  fv  —  f\  — >•  0,  as  v  — >•  00,  then  there  exists  an  v  such  that  for  all  v  >9, 

e((e,5)-argmin{/o  +  iff<0}  C  Bp,  (/?,  a)-argmin{/0  +  t/<-7}) 

<sup Bp  |/o  -  /o  |  +  max{l,K(p')}supBp  \fv  -  f\. 


Proof.  There  exists  an  u  such  that  for  all  v  >  u,  supBp  |/q  —  /o|  <  (/?  —  e)/4  and 


sup Bp  I  f 


f  |  <  min 


p'  —  p  a  —  6  —  7 
2  ’  2 


4max{l, k(//)}  /  " 


Let  v  >  v.  We  start  with  an  application  of  Theorem  4.6  with  /q  and  fv  in  the  role  of  go  and  g, 
respectively,  and  7  =  7  and  that  theorem’s  7  being  set  to  {(5  —  e)/4.  This  results  in 


e((e,  (5)-  argmin{/o  +  if»<o}  fl  Bp,  (e  +  7+  +  (/3  -  e)/4  +  (/3  -  e)/4,  a)-  argmin{/0  +  t/<-7})  <  7+>  (6) 
where  ?y+  =  77+ (/o  +  t/<a_7;  /q  +  iff <5).  Next,  we  invoke  Theorem  4.5  and  conclude  that 

e(lev,5  /"  n  leva_7  /)  <  7+  (/;  /")  <  supBp  \f  -  f\< 
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because  a  —  7  >  S  +  (a  —  6  —  7 )/2  >  5  +  Finally,  we  bring  in  Proposition  4.7.  Since 

p  +  e(lev^  fv  H  fBp,  levo,_7  f)<p  +  (p'  —  p)/2  <  p'\  we  conclude  that  the  present  choice  of  f /  suffices  in 
that  proposition  and  a  there  is  set  to  a  —  7  >  0.  Thus, 


V+  <  supBp  |/o  -  /o|  +  max{l,  K(p')}e(\evs  fu  n  Bp ,  levQ_7  /) 
<  sup^  |/o  -  /o|  +  max{l,  n(p')}  supB  |  f  -  f\ 


(7) 


<  +  max{l,  n(p')} 


(3  —  £ 


4max{l,  n(p')} 


< 


/3  —  £ 


Thus,  e  +  r)+  +  (rj  —  e)/4  +  (77  —  e)/4  <  (3  and  we  see  from  (6)  and  (7)  that  the  conclusion  holds.  □ 

We  note  that  the  proposition  makes  a  statement  about  rate  of  convergence  of  near-optimal  near- 
feasible  solutions  of  the  approximate  problem  min{/g  +  if*  < 0}  to  solutions  of  a  slightly  restricted 
“original”  problem  min{/o  +  i/<_7},  with  7  >  0  arbitrarily  small.  The  use  of  such  a  restriction  allows 
us  to  avoid  possibly  hard-to-verify  conditions  on  the  constraint  function  and  its  level  sets. 


5  Epi-Splines  and  Construction  of  Approximations 

In  the  previous  sections,  we  bounded  the  aw-distance  between  two  given  lsc  functions  and  related  such 
bounds  to  solution  estimates  for  the  minimization  problems  defined  by  those  functions.  We  now  turn 
to  the  construction  of  a  function  that  approximates  a  given  lsc  function.  In  practice,  approximations 
of  optimization  problems  depend  on  the  nature  of  the  application.  We  take  an  abstract  perspective 
and  examine  piecewise  constant  functions  that  resemble  the  simple  functions  of  integration  theory  and 
(zeroth-degree)  polynomial  splines  from  functional  approximation  theory.  The  approximating  functions 
are  defined  by  a  finite  number  of  parameters.  As  we  see  below,  they  can  be  made  to  approximate  to 
an  arbitrary  level  of  accuracy  any  functions  in  lsc- fens (X)  under  some  assumptions  on  X,  relying  on 
epi-convergence  and  the  aw-distance  to  formalize  the  meaning  of  accuracy.  The  results  in  this  section 
certainly  open  up  computational  possibilities  for  solving  difficulty  optimization  problems,  but  also 
provide  new  means  to  establish  theoretical  results  about  lsc  functions  through  their  finite-dimensional 
approximations . 

We  adopt  the  notation  cl  A  and  int  A  for  the  closure  and  interior,  respectively,  of  a  subset  A  of  a 
topological  space.  The  approximating  functions  are  defined  in  terms  a  finite  collection  of  subsets  of  X. 

5.1  Definition  (partition)  A  finite  collection  Ri,  R2, ...,  Rn  of  open  subsets  of  X  is  a  partition  of  a 
closed  set  B  C  X  if  cl  Ru  =  B  and  R *.  n  Ri  =  0  for  all  k  l. 

For  any  /  :  X  — >•  1R  and  x  G  X,  let  liminf X'^xf{x')  :=  lim^o  hif^/gs^^)  f(x').  Clearly,  /  is  lsc  if 
liminfj./^.j.  f{x')  >  f(x)  for  all  x  €  X  (see  for  example  [3,  Section  2]).  The  approximating  functions, 
called  epi-splines,  are  defined  next. 

5.2  Definition  (epi-splines)  An  epi-spline  s  :  X  — >•  1R,  with  partition  TZ  =  {Rk}k=i  °7  a  closed  set 
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B  C  X,  is  a  function  that 

on  each  Rk,  k  =  1, N,  is  constant, 

has  s(x )  =  oo  for  x  0  B,  and  for  every  x  €  X,  has  s(x)  =  s(x'). 

The  family  of  all  such  epi-splines  is  denoted  by  e-spl(T^). 

This  definition  straightforwardly  extends  from  lRn  to  general  metric  spaces  the  one  in  [27].  There  we 
also  deal  with  “higher-order”  epi-splines  involving  polynomials  of  degrees  greater  than  zero  on  each  Rk, 
which  motivates  the  reference  to  “splines”  in  the  name.  The  same  possibility  exists  here,  but  we  shy 
away  from  that  subject  due  to  the  complications  related  to  extending  polynomials  to  general  metric 
spaces.  The  reference  to  “epi”  in  the  name  is  motivated  by  the  choice  of  epi-convergence  as  the  notion 
of  convergence  as  we  see  below. 

Clearly,  by  definition,  every  epi-spline  is  lsc.  The  ability  of  epi-splines  to  approximate  arbitrary  lsc 
functions  relies  on  a  refinement  of  the  partition. 

5.3  Definition  (infinite  refinement)  A  sequence  \JZU}^=1  of  partitions  of  a  closed  set  B  C  X,  with 
IZU  =  [Rk}k=1,  JS  an  infinite  refinement  of  B  if 

for  every  x  €  B  and  £  >  0,  there  exist  u  G  IV  and  S  G  (0,  s)  such  that 
Rfi,  C  IB(x,  e)  for  every  is  >  v  and  k  satisfying  Rk  n  JB(x,  6)  0. 

We  note  that  this  notion  of  refinement  is  local  in  nature,  which  is  essential  as  we  aim  to  address  partitions 
of  unbounded  sets.  A  sufficient  condition  for  the  existence  of  an  infinite  refinement  is  separability. 

5.4  Proposition  (existence  of  infinite  refinement)  If  B  C  A  is  nonempty,  separable,  and  solid3,  then 
there  exists  an  infinite  refinement  of  B. 

Proof.  Let  x°  E  B  and  Bu  =  fB(x°,  is)  n  B,  is  £  IN.  The  separability  of  B  implies  that  there  exists  a 
sequence  {IB(x3,  £3)}jeN,  with  {x3}jeN  a  dense  subset  of  B  and  {e 3}jeN  a  dense  subset  of  (0,oo)  such 
that  {{x3 ,  eJ)}j£]y  is  dense  in  B  x  (0,  oo)  under  the  product  topology.  For  every  is,  the  boundedness  of 
Bu  implies  that  there  exists  a  Ju  <  oo  such  that  Bu  C  {lB{x3 ,e3)}J^=1.  Let  {Mu}u£n  be  a  sequence  of 
scalars  that  tend  to  infinity  and  Mu  >  Jv . 

We  are  then  ready  to  construct  the  open  sets  that  form  the  partitions,  which  subsequently  will 
be  shown  to  be  an  infinite  refinement.  For  every  is  the  process  is  identical.  First,  sort  the  balls 
{]B(x3,  £3)}j£[  in  the  order  of  nondecreasing  radii  and  let  this  ordered  set  be  {IB(x3k,  e^fc)}^.^,  i.e., 
s]k  <  e*k+i  for  all  k  =  1, ...,  AIU  —  1.  Second,  set  R\  =  in t(Bu  n  1B(x3i  ,e3 1 ))  and  recursively 

R\  =  int  {^Bv  C  B{x?*,S))  \  ufr/Af)  for  k  =  2, 3, ..., Mv . 

Set  Nv  =  Mu  +  1  and  let  RvNV  =  int (B  \  1B(x°,  is)).  We  observe  that  some  Rk  maybe  empty,  but  that 
is  immaterial.  Obviously,  Rk,  k  =  1,  ...,NU,  are  open  and  nonoverlapping,  and  U klxRk  C  B.  Since  B 

3B  is  solid  if  B  =  cl  (inti?) 
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is  closed,  we  also  have  that  U^T1  c\Rk  C  B.  We  can  conclude  that  Rk,  k  =  1,  ...,NU,  is  a  partition  of 

B  after  establishing  that  U^T1  cl  Rk  D  B.  Suppose  that  x  £  B.  Since  B  =  cl(int-B)  (i.e.,  is  solid),  there 

exist  and  positive  numbers  such  that  — >•  x,  <5A‘\0,  and  IB(y^,5^)  C  int  B .  For 

every  /j,  there  is  a  £  {1,  ...,NU}  such  that  IB(y^,  <5’#i) n Rk  /  0  as  we  see  from  the  construction.  Hence, 

there  exists  z M  <5M)  Hi?))  ,  which,  due  to  5^X0,  tend  to  x.  This  implies  that  x  €  U^T1  c\Rk  and 

therefore  {Rk}k=i  is  a  partition  of  B.  This  holds  for  all  v. 

We  next  show  that  Definition  5.3  holds.  Let  x  £  B  and  e  >  0.  There  exists  a  u  >  d(x°,x)  +  s 

such  that  the  collection  {JB(xi , contains  a  ball  IB (xJ* , eJ"* )  D  JB{x,e/A)  and  eJ*  <  e/3.  Suppose 

that  this  ball  is  number  k *  in  the  sorted  collections  {IB(xR ,  e-?fc  jj/lj ,  ^  =  F,  F  +  1, ....  Consequently, 

U^1cli?^  D  1B(x,e/ 4)  for  all  v  >  v.  By  construction  C  IB (x]k ,  e-T )  for  all  k  and  za  Thus,  for 

every  v  >  v  and  k  =  1  ,  sup d(y,y' )  <  e/3  due  to  the  nondecreasing  radii  of  the  balls  in 

the  sorted  collections.  We  therefore  have  that  Rk  C  IB(x,  e)  for  every  k  =  1, ...,  k *  and  n  >  n  satisfying 

hy 

Rk  n  IB(x,  e/4)  /  0.  Since  from  above  we  know  that  Ufc^1  c\Rk  D  lB(x,e/&)  for  all  v  >  v,  we  conclude 
that  n  IB(x,  e/4)  =  0  for  all  k  =  k”  +  1,  fc*  +  2, ...,  A1'  and  u  >u.  Consequently,  Definition  5.3  holds 
with  6  =  e/4.  □ 

The  above  proof  provides  guidance  towards  the  construction  of  infinite  refinements,  for  which  there 
are,  of  course,  many  possibilities.  A  main  approximation  results  for  epi-splines  is  given  next,  where 
the  approximation  is  in  the  sense  of  epi-convergence  and  pointwise  convergence.  The  result  is  an 
improvement  over  one  in  [27]  by  allowing  A  to  be  a  general  metric  space,  not  only  Mn .  and  by  also 
establishing  pointwise  convergence  as  well  as  an  upper  bound.  Later,  we  give  a  stronger  conclusion  of 
convergence  in  the  sense  of  the  aw-distance  under  additional  assumptions. 

5.5  Theorem  (approximation  of  lsc  functions)  If  is  an  infinite  refinement  of  a  nonempty 

closed  set  B  C  X,  then  for  every  f  £  lsc-fcns(B)  there  exist  epi-splines  su  £  e-spl(TvT)  satisfying  the 
following: 

(i)  sv  epi-converges  to  /, 

(ii)  su  converges  pointwise  to  f  on  X,  and 

(in)  su(x)  <  max{-i/,  f(x)}  for  all  v  and  x  £  X. 

Proof.  Let  /  £  lsc-fcns(H)  and  1ZV  =  For  every  v  £  IN  and  k  £  {1,  2, ...,  Nu},  we  let 

c 7vk  =  min  jz/,  max  |-z^,  infci  R»  /|| 

and  construct  sv  :  X  ->  1R  as  follows: 

su(x)  =  ^  mi  11^  | ak  :  x  €  cli?))|  for  x  £  B,  and  su(x)  =  oo  for  x  0  B. 

For  every  v  £  IV,  the  open  sets  Rk,  k  =  1 ,...,  TV",  are  disjoint.  Thus,  if  x.  x'  £  Rk,  then  sv(x)  = 
su(x')  =  ak,  and  su  is  constant  on  Rk.  If  x  £  B,  but  not  in  Rk  for  any  k  =  1, ...,  Nu,  then  for  the  set 
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Ku(x)  =  {k  :  x  G  cl Rk}  C  {1,  ...,NU}  we  have  that  sv{x)  =  min{cr£  :  k  G  iF"(x)}.  Moreover,  there 
exists  5"  >  0  such  that  (U kgio'ix)  cl-R^)  FI  dB(x,5p)  =  0.  This  implies  that  lim5^0  ^  sv(x')  = 

su(x),  which  establishes  that  sv  €  e-spl(7xT). 

We  next  show  epi-convergence  of  su  to  /.  Let  x  G  B  be  arbitrary.  By  the  lsc  of  /,  for  every  e  >  0 
there  exists  5  >  0  such  that  f(x')  >  /(x)  —  £  whenever  x'  G  lB(x,d).  Consequently, 

avk  >  ruin  jz/,  inf ci  rv  /}  >  min{/y,  f(x )  —  e}  when  R k  C  fB(x,  5). 

Since  {7^Z>}^C=1  is  an  infinite  refinement,  there  exists  a  9  and  do  G  (0,5)  such  that  R k  C  fB(x,d )  for 
every  is  >  9  and  k  satisfying  Rvk  n  fB{x, So)  /  0-  Consequently,  for  5o/2)  and  is  >  9, 

sv{ x')  =  rnin^  jo-))  :  x'  G  cli?^|  >  min{ za  /(x )  —  e} 

because  every  k  and  is  >  9  with  xy  G  cli?))  must  also  have  Rk  C  lB(x,5).  For  x'  G  JB(x,do/2)  \ 

B ,  su(x')  =  oo.  These  fact  establishes  that,  for  every  sequence  x"  — >•  x  G  B,  liminf  su(xu)  > 
liminf  min{z/, /(x)  —  e}  =  f(x)  —  e,  where  the  possibility  that  /(x)  =  oo  is  included.  Since  e  is  ar¬ 
bitrary,  liminf  sv(xv)  >  /(x).  This  inequality  also  holds  for  x  0  B  also  because  B  is  closed  and 
sv  =  /  =  oo  on  X  \  B.  Next,  again  let  x  G  B  be  arbitrary.  By  construction, 

sv{x)  <  <rk  <  max{  —  is,  infclR£  /}  for  every  k  satisfying  x  G  cl  Rk. 

Hence,  su(x)  <  max{— is,  /(x)}.  Set  xv  =  x  for  all  is  and  we  obtain  that  limsup  sv{xu)  =  lirnsup  sv{x)  < 
/(x).  If  x  0  B,  then  /(x)  =  oo  and  the  previous  inequality  holds  trivially  for  any  xv .  Thus,  we  have 
established  epi-convergence  of  su  to  /. 

Since  liminf  su(x)  >  f(x)  holds  by  virtue  of  the  established  epi-convergence,  we  also  have  that 

sv(x)  — >•  /(x)  for  all  x  G  X,  which  establishes  the  pointwise  convergence.  The  fact  that  sv(x)  < 

max{-i/,  /(x)}  for  all  x  G  X  is  settled  already.  □ 

The  proof  of  Theorem  5.5  is  constructive.  Given  a  partition  {Rk}k=i,  an  approximating  epi-spline 
to  a  lsc  function  /  is  essentially  the  piecewise  constant  function  given  at  x  G  Ry-  by  inf^fc  /. 

We  next  examine  approximation  of  functions  in  lsc- fens  (A*)  by  epi-splines  in  the  sense  of  the  aw- 
distance,  which  requires  us  to  adopt  a  “uniformity”  requirement  on  infinite  refinements.  As  we  see 
below,  this  imposes  further  restrictions  on  the  underlying  space  (X,  d). 

5.6  Definition  (locally  uniform  infinite  refinement)  A  sequence  of  partitions  of  a  closed  set 

B  C  X,  with  1ZU  =  {Rk is  a  locally  uniform  infinite  refinement  of  B  if 

for  every  x  G  B,  p  >  0,  and  e  >  0,  there  exists  a  9  G  IV  such  that 

Rf  C  lB(y,  e)  for  every  y  G  lB(x,  p)  n  B,is  >  9,  and  k  satisfying  y  £  cl  Rk. 

It  should  be  apparent  that  a  locally  uniform  infinite  refinement  is  also  an  infinite  refinement.  A  locally 
uniform  infinite  refinement  needs  to  have  9  that  applies  not  only  at  a  single  point  x,  as  in  the  case  of 
an  infinite  refinement,  but  for  all  points  in  arbitrarily  large  balls.  Naturally,  compactness  ensures  such 
a  property  as  established  next. 
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5.7  Proposition  (sufficient  condition  for  locally  uniform  infinite  refinement)  There  exists  a  locally 
uniform  infinite  refinement  of  every  nonempty  solid  set  B  C  X  for  which  B  nfB(x,  r )  is  compact  for  all 
x  E  B  and  r  >  0. 

Proof.  Let  x°  E  B,  v  E  IV,  and  Bv  =  IB(x°,  u)  fl  B.  Since  Bu  is  compact,  there  exists  Mv  <  oo  and 
{xV'k}k= 1  c  such  that  U^1lB(xl/,k,  \/v)  D  Bv .  Set  R\  =  int(f?I/  n  fB(xv'1,  1/z/))  and  recursively 

R^  =  int  ((5"  n  lB(xL,’k,  1/z/))  \  ufri1/?)/)  for  k  =  2, 3, M". 

Set  IV"  =  Mv  +  1  and  let  =  int  (5  \  IB{x °,  z/)).  We  observe  that  some  Rk  maybe  empty,  but  that 
is  immaterial.  Obviously,  Rk,  k  =  1,  ...,NU,  are  open  and  nonoverlapping,  and  U k=xRk  C  B.  Since  B 
is  closed,  we  also  have  that  U^)1  cl  Rk  C  B.  We  conclude  that  cl  RkDB  after  following  the  same 
argument  as  in  the  proof  of  Proposition  5.4.  Therefore,  {Rk}kl^  is  a  partition  of  B. 

We  next  show  that  Definition  5.6  holds.  Let  x  £  B,  p  >  0,  and  e  >  0.  Set  v  equal  to  the  smallest 
integer  no  smaller  than  max{2/e,  d(x ,  x°)+e+/j}.  By  construction,  for  every  u  >  u  and  k  =  1, ...,  Nv  —  1, 
sup {d(y,y')  :  y,y'  E  Rk}  <  e.  Moreover,  for  such  v,  fB(y,e)  n  B  C  Bu  for  all  y  E  fB(x,p)  n  B  and 
therefore  Rk  C  fB(y,e)  whenever  y  E  cl  Rk  and  y  E  JB(x,p)  n  B.  This  establishes  the  result.  □ 

Next  we  strengthen  Theorem  5.5  by  considering  the  aw-distance.  We  say  that  an  epi-spline  s  on 
X  is  rational  if  s(x)  is  a  rational  number  for  every  x  E  doms.  The  subset  of  rational  epi-splines  in 
e-spl(7£)  is  denoted  by  r-spl(7L). 

5.8  Theorem  (rational  epi-splines  dense  in  lsc  functions)  If  {1ZU}^=1  is  a  locally  uniform  infinite 
refinement  of  a  nonempty  closed  set  B  Cl,  then 

OO 

[J  r-spl(7L,y)  is  dense  in  (lsc-fcns(B),  d). 

V=1 

Proof.  Let  /  E  lsc-fcns(B)  and  TZU  =  For  every  v  E  IN  and  Rk,  k  =  1,2 ,...,NV,  with 

infci^  /  finite,  let  qk  be  a  rational  number  in  [infci  f  —  l/n,  infci  /].  Let  ak  =  min  {n,  max{-g  Qk}}- 
For  v  E  IV  and  Rk ,  fc  =  1,2, ...,  Nv ,  with  infci  ji"k  /  =  oo,  set  ok=n  and  for  inf cj /  =  —  oo  set  ak  =  —v. 
We  proceed  by  constructing  sv  :  X  — >•  IR  as  follows: 

sv(x)  =  ^  mi  11^  | ok  :  x  E  cl  f?)(j  for  iEB,  and  sv{x)  =  oo  for  x  0  B. 

A  replication  of  the  arguments  in  the  proof  of  Theorem  5.5  establishes  that  su  E  r-spl(AT). 

We  next  show  that  for  any  p  >  0,  d,p(su,f)  — >  0  as  v  — >•  oo.  By  construction,  if  infci^£  /  is  finite, 
then 

sv{x)  <  crk  <  max  {  —  z pqk}  <  max  {  —  v,  infci^  /}  <  max{— v,  f(x)}  when  x  E  cl  Rk. 

If  inf'd  rq  f  =  Too,  then  sv{x)  <  v  for  x  E  cl  Rk.  Since  this  holds  for  all  k  =  1,...,AT,  sv{x)  < 
max{— z/,  f(x)}  for  x  E  X.  For  u  >  p,  x  E  X ,  and  q  =  0, 

infB0r,77)  =  ^(x)  <  max{/(i),  — z^}  +  q  <  max{/(i),  -p}  +  q. 
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Therefore  even  77  =  0  satisfies  the  second  collection  of  constraints  in  the  definition  of  dPp(sv ,  /).  We  now 
turn  to  the  other  constraints  in  the  definition  with  the  roles  of  sv  and  /  reversed.  Let  e  >  0  and  z  €  B. 
Since  is  a  locally  uniform  infinite  refinement,  there  exists  a  G  JV  such  that  R k  C  IB{x,e ) 

for  every  x  €  B  n  lB(z,d(x,  z )  +  p),  v  >  P,  and  k  satisfying  x  G  cl  Rvk.  Let  x  G  lev^s"  n  JBp  and 
v  >  max{p,  P}.  Since  d(x,  z)  <  d(x,  x)  +  d(x,  z)  <  p  +  d(x,  z),  x  G  B  n  JB(z,  d(x,  z)  +  p).  There  exists  a 
k x  G  {1, Nv }  such  that  x  G  clf?)L  and  infd/j^  /  <  sv{x)  +  l/u  due  to  the  fact  that  sv(x)  <  p  <  v. 
Since  k x  is  one  of  possibly  several  k  for  which  Rk  C  lB(x,  e)  holds,  we  obtain  that  for  v  >  max{p,  v,  1/e}, 

infB(x,e)  /  <  infciR-  /  <  sv(x)  +  £<  max{sz/(x),  -p}  +  e. 

Consequently,  the  first  collection  of  constraints  in  the  definition  of  dPp{sv ,  /)  is  satisfied  with  77  =  e.  We 
then  have  that  (i0p{sv ,  /)  <  e  for  all  v  >  max{p,  9, 1/e}.  Since  £  >  0  is  arbitrary,  we  conclude  that 
d°p(su,  f)  — >  0  as  v  — >•  00.  By  Propositions  3.1  and  3.2,  this  suffices  to  establish  that  dL(su,  f)  — >•  0.  □ 

Next,  we  give  a  rate  of  convergence  result  for  epi-splines,  which  is  novel  even  for  lRn.  The  error  in 
approximation  of  a  lsc  function  by  an  epi-spline  is  bounded  by  the  “size”  of  the  open  sets  making  up 
its  partition.  Specifically,  for  a  partition  1Z  =  {i?/c}'^=1  of  a  nonempty  closed  set  B  C  X  and  p  >  0,  we 
define  its  meshsize  as 

mp( TV)  :=  inf  {77  >  0  :  Rk  C  ]B(x,  r/)  for  all  x  G  B  fl  1BP  and  k  satisfying  x  G  cl  Rk)- 

5.9  Theorem  (rate  of  convergence  of  epi-splines)  For  a  partition  7Z  =  {Rk}k=i  0 f  a  nonempty  closed 
set  B  C  X  and  p  >  0,  we  have  that 

for  every  f  G  lsc-fcns(B),  there  exists  an  s  G  e-spl(7£)  such  that  d°p(s,  f)  <  mp(lZ). 

If  rnp(lZ)  >  0,  then  s  can  he  selected  to  he  rational. 

Proof.  Let  /  G  lsc- fens (B).  We  start  with  the  case  of  mp(JZ )  >  0  and  set  v  >  rna x{mp(lZ),  p}.  For 
every  R *.,  k  =  1, 2, ...,  N,  with  infci^fc  /  finite,  let  q ^  be  a  rational  number  in  [infci#fc  f —mp(IZ),  inf c\Rk  /]. 
Moreover,  let  Ok  =  min{n,max{— n,  qk}}-  For  Rk,  k  =  1,2,  ...,1V,  with  infci#fc  /  =  00,  set  Ok  =  v  and 
for  infci  nk  f  =  —00  set  Of;  =  —P.  We  proceed  by  constructing  s  :  X  — >•  1R  as  follows: 

s(x)  =  ^  min  ^  | :  x  G  clf?fc|  for  x  G  B,  and  s(x)  =  00  for  x  0  B. 

By  the  same  arguments  as  in  the  proof  of  Theorem  5.5,  we  conclude  that  s  G  r-spl(7£). 

We  next  establish  the  approximation  error  associated  with  s.  Mimicking  the  lines  of  reasoning  in 
the  proof  of  Theorem  5.8,  we  obtain  that  miB/x  v\  s  <  max{/(.x),  —  p}  +  r]  holds  with  r]  =  0  for  all 
x  €  X.  Next  we  reverse  the  roles  of  s  and  /.  Let  x  G  levp  s  fl  lBp.  Certainly,  all  k  with  x  G  cl  Rk  has 
Rk  C  lB(x,mp(TZ)).  There  exists  a  kx  such  that  x  G  cl Rkx  and  infci#^  /  <  s(x)  +  mp(JZ)  due  to  the 
fact  that  s(x)  <  p  <  P.  Since  kx  is  one  of  possibly  several  k  for  which  Rk  C  IB{x,mp{jVj)  holds,  we 
obtain  that 


infB(a;,mp(7e))  /  <  infciRfciE  /  <  s{x)  +  mp{Jl)  <  max{s(x),  -p}  +  mp(n). 
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We  can  therefore  conclude  that  d®(s,  /)  <  rnp (77)  when  mp(IZ)  >  0.  If  rn p(TZ)  =  0,  the  same  arguments 
hold  except  that  could  be  irrational  and  s  G  e-spl(77).  □ 

An  example  illustrates  Theorem  5.9.  In  optimization  over  X  approximations  might  arise  from 
approximations  of  X  by  a  simpler  set,  say  X.  As  seen  in  Proposition  3.3,  the  error  introduced  by  this 
approximation  is  largely  given  by  dp(X,X).  To  make  this  concrete,  let  (X,d)  =  (lsc-fcns (Mn),d)  and 
77  be  a  partition  of  lRn  that  defines  the  simpler  set  X  =  e-spl(77).  We  recall  from  Propositions  3.1  and 
3.2  (applied  with  X  =  IRn  and  x  =  0)  that  for  /  G  lsc-fcns  (J?n)  and  s  G  e-spl(77), 

d(s,  f)  <  (1  -  e_7)(Jy (s,  /)  +  e“7[max{<5s,  Sf}  +  7  +  1]  for  any  7  >  0, 

where  6S  =  dist(0,  epi  s),  Sf  =  dist(0,  epi  /),  and  7'  >  27  +  max{hs,  Sf}.  For  every  /  G  lsc-fcns (J?n) 
there  exists  by  Theorem  5.9  an  sf  G  e-spl(77)  such  that  dy(sf,f )  <  my  (77).  Moreover,  there  exists  a 
5  >  sup{max{<5S/,  Sf}  :  f  G  lsc-fcns( J?n)  n  1BP}-  For  every  7  >  0  and  /  G  lsc-fcns(l?n)  fl  lBp,  we  have 
then 

dist(/,  e-spl(fc))  <  d(sf,  /)  <  (1  -  e~1)m2l+s(JZ)  +  e— 7[<5  +  7  +  1]. 

Thus, 

clp (e-spl(7£),  lsc-fcns(J?"))  <  (1  —  e_7)ni27+i(K)  +  e~1[5  +  7  +  1]. 

We  end  the  section  with  an  observation  that  the  existence  of  a  locally  uniform  infinite  refinement 
is  intimately  tied  to  compactness  of  balls;  Proposition  5.7  shows  that  such  compactness  is  a  sufficient 
conditions.  The  fact  that  it  is  also  necessary  when  B  is  complete  is  stated  next. 

5.10  Proposition  (necessary  condition  for  locally  uniform  infinite  refinement)  If  there  exists  a  locally 
uniform  infinite  refinement  of  a  nonempty  closed  set  B  Cl  and  B  is  complete,  then  every  closed  ball 
in  X  must  have  a  compact  intersection  with  B. 

Proof.  By  Theorem  5.8,  the  assumption  implies  that  (lsc-fcns (B),d)  is  separable.  In  view  of  [10, 
Theorem  3.1.4]  we  know  that  this  takes  place  if  and  only  if  closed  and  bounded  subsets  of  ( B,d )  are 
totally  bounded.  Thus,  closed  balls  in  X  intersected  with  B  are  totally  bounded.  Their  compactness 
then  follows  from  the  assumption  of  completeness.  □ 


Appendix 

Proof  of  Proposition  3.1.  Item  (i)  follows  immediately  from  the  definitions.  For  (ii),  first  observe 
that  for  a  nonempty  set  C  C  X  x  1R  and  x,y  £  X  x  M  we  have  for  any  zGC,  dist(x,  C )  <  d(x,  z)  < 
d(x,  y)  +  d(y,  z).  Minimizing  over  z  G  C,  we  obtain  that  dist(x,  C )  <  d( x,  y)  +  dist(y,  C)  and  thus 

|  dist(x,  C)  —  dist(y,  C)|  <  d(x,  y). 

Second,  in  the  notation  h(x)  =  \  dist(x,  C)  —  dist(x,  D)  \  for  nonempty  C,  D  C  X  x  1R,  we  have  that 

\h(x)  —  h(y)\  <  |[dist(x, C)  —  dist(x, D)\  —  [dist(y, C)  —  dist(y, T>)]| 

<  |  dist(x,  C)  —  dist(y,  (7)1  +  1  dist(x,  D)  —  dist (y,  D)\  <  2 d(x,  y). 
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Let  e  >  0.  For  every  y  €  Sp/,  there  exists  an  x  €  §p  such  that  d(x,  y)  <  e(Sp/,  §p)  +  e.  Combining  these 
last  two  facts  and  replacing  C  and  D  by  epi-graphs,  we  obtain  that 

dp>{f,g)  =  sup{|dist(y,epi/)  -  dist(£/,  epi#)|  :  y  €  §p'} 

<  sup{|  dist(x,  epi  /)  —  dist  (a;,  epi  <?)|  +  2e(Sp/,  Sp)  +  £  :  x  G  §p}  =  dp(f,  5)  +  2e(Sp/,  Sp)  +  e, 

which  establishes  (ii)  after  realizing  that  e  >  0  is  arbitrary. 

Next  consider  (iii).  Suppose  that  C,DcIxE  are  nonempty  closed,  e  >  0,  and  p  >  0.  We  first 
show  that 

dist (- ,  D)  <  dist(-,  C)  +  £  on  §p  implies  that  C  n  Sp  C  Df  :=  {x  €  X  x  1R  :  dist(x,  D )  <  e}. 

The  claim  is  trivial  if  C  fl  Sp  is  empty.  For  nonempty  C  fl  Sp,  we  have  for  every  x  €  C  FI  Sp  that 
dist(x,D)  <  e  and  the  implication  follows.  The  translation  of  this  fact  to  the  context  of  epi-graphs 
establishes  the  lower  bound  in  (iii).  Second,  we  establish  that 

C  n  §p/  C  Df  implies  dist (-,  D)  <  dist (- ,  C)  +  e  on  §p  for  p  >  2p  +  dist((x,  0),  C).  (8) 

For  2  G  (7n§p<  C  Df  and  x  €  X  x  1R,  dist(x,  D)  <  d(x,  z)  +  dist(z,  D)  <  d(x,  z)  +  e.  The  minimization 
over  z  €  C  fl  Sp/  gives  that 

dist(x,  D)  <  dist(x,  C  n  §p/)  +  e.  (9) 

This  holds  trivially  if  CnSp/  =  0.  Suppose  that  x  €  §p  and  p'  >  2p  +  dist((.x,  0),  C ).  For  every  v.  there 
exists  yv  €  C  such  that  d(x,yu)  <  dist(.x,C')  +  l/v.  Moreover, 

d((x,  0),  yu)  <  d((x,  0),x)  +  d(x,  yu)  =  d((x,  0),  x)  +  dist(x,  C)  +  \/v 

<  d((x,  0),  x)  +  d(x ,  (. x ,  0))  +  dist ((5,  0),  C)  +  \/v  <  p  +  p  +  dist((.x,  0),  C)  +  1/v. 

Since  p'  >  2p  +  dist((x',  0),  C),  there  exists  a  u  such  that  yv  €  C  fl  Sp/  for  all  v  >  v.  For  such  v, 

dist(x,  C  Fl  Sp/)  <  d(x,  yv)  <  dist(x,  C)  +  1/v. 

Letting  u  — >•  00  in  this  expression  and  observing  that  dist(x,  C  n  Sp')  >  dist(x,  C )  generally,  we  obtain 
that  dist(x,  CnSp')  =  dist(x,  C ),  which  together  with  (9)  establishes  (8).  The  implication  in  (8)  directly 
confirms  the  upper  bound  in  (iii).  If  ( X ,  d)  is  proper,  then  in  view  of  Lemma  2.4  we  can  take  yv  above 
to  satisfy  d{x,yv)  =  dist(x,  C)  for  all  u.  Thus,  the  need  for  the  \/v  term  vanishes  and  the  stronger 
statement  given  at  the  end  of  the  theorem  is  established. 

Item  (iv)  follows  trivially  from  the  definition  of  dLp.  For  (v)  and  (vi),  we  follow  the  lines  of  arguments 
in  the  proof  of  [24,  Lemma  4.41].  Clearly, 

rp  poo 

d(f,g)=  /  dLT(f,  g)e~Tdr  +  /  dT(f,  g)e~Tdr. 

Jo  J  p 

Since  dT(f,g)  is  nondecreasing  as  r  increases,  we  have  that 

do(f,g)  [  e~T dr  <  [  dT(f,g)e~TdT<dLp{f1g)  f  e~Tdr. 

Jo  Jo  Jo 
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and 


/*oo  poo  poo 

dP(f,g )  /  e~Tdr  <  /  dT(f,  g)e~Tdr  <  /  [max{5/,  59}  +  r]e^rdr, 

J  p  J  p  J  p 

where  the  last  inequality  follows  (iv).  Carrying  out  the  integrations  on  the  left-  and  right-hand  sides, 
we  obtain  (v)  and  (vi). 

The  lower  bound  in  (vii)  is  obtained  by  letting  p  tend  to  infinity  in  (v).  Item  (vi)  with  p  =  0 
furnishes  the  upper  bound.  □ 

Proof  of  Proposition  3.2.  We  start  by  establishing  the  finiteness  of  dp(f,  g)  for  any  p  >  0.  Since  f,g£ 
lsc-fcns(A),  there  exist  x,  y  €  X  such  that  g(x),  f(y )  <  oo.  Let  fj  =  p+vna,x{d{x,  x),  d(x,  y),  g(x),  f(y)}, 
which  is  finite.  Then,  for  x  €  1  evp/  n  lBp,  d(x,x)  <  d(x,x)  +  d(x,x)  <  p  +  (fj  —  p)  =  fj  and 
thus  inf  B(x,fj)  g  —  g{%)  <  V  ~  P  <  max{/(x),  — p}  +  fj.  Likewise,  for  x  €  levp<7  n  1BP,  inf  f  < 
max{<7(x),  —p}  +  fj.  Thus,  d,p(f,g)  <  fj,  which  establishes  the  rightmost  inequality. 

Obviously,  dp(f,g)  =  inf { 77  >  0  :  dp{f,g)  <  77},  which  in  view  of  the  minimization  taking  place  in 
the  definition  of  dp(f,g )  motivates  the  examination  of  the  relation  dp(f,g)  <  77  for  77  >  0.  It  follows 
directly  from  the  definition  of  dp  that 

dp{f,  g)  <  77  if  and  only  if  dist(x,  epi  77)  <  77  Vx  €  epi  /  D  §p  and  dist(x,  epi  /)  <  77  Vx  €  epi  77  fl  Sp, 
which  in  turn  is  equivalent  to  having 

( epi  /)  O  §p  C  (g)  and  ( epi g)  nSpCh| (/), 

where  D+(f)  :=  {x  £  X  X  ]R  :  dist(x, epi/)  <  77}  and  similarly  for  D+(g).  By  virtue  of  being 
defined  in  terms  of  distance  to  an  epigraph,  we  have  that  (x,  7/0)  €  D+  (77)  implies  (x,xo)  €  D+(g)  for 
all  xo  >  2/0-  Thus, 

(  epi  /)  O  Sp  C  D1 j~ ( g )  if  and  only  if  (x,  fp(x))  €  Dj J" (77)  for  all  x  €  dom  fp, 

where  the  function  fp  :  X  — >  J?.  is  given  by  /p(x)  =  max{/(x),  — p}  if  x  €  levp  /  n  !Bp  and  /p(x)  =  00 
otherwise.  By  definition,  a  point  (x,xo)  €  D+(g)  if  and  only  if  dist((x,  xo),  epi g)  <  77.  The  latter 
condition  is  more  explicitly  stated  as 

inf  |  max{d(x,  y),  |x0  -  yo|}  :  g(y)  <2/0  ,V  £  X,  y0  <E  J?|  <  77. 

We  are  now  in  a  position  to  establish  the  lower  bound  and  let  5  >  0.  Collecting  the  above  facts,  we 
find  that  if  dp(f,  g)  <  77,  then 

inf  |  max{d(x,  y),  |/p(x)  -  2/o|}  :  g(y)  <2/0 ,!/£  X,  y0  G  J?  j  <  77  for  x  €  dom  fp. 

Let  x  G  dom/p.  Hence,  for  every  e  €  (0,5],  there  exists  (y£,yo£)  £  X  x  R  such  that  g(ys)  <  2/0e> 
ye)  <  77  +  e,  and  |/p(x)  -  y0e|  <  77  +  e.  Consequently,  g(y£)  <  fp(x)  +  77  +  e  and  y£  €  B(x,  77  +  e). 
Moreover,  inf^rx^+tf)  5  <  inf^(XjJ?+e)  77  <  /p(x)  +  77  +  e.  Since  this  relation  holds  for  all  e  G  (0,5], 
infB(X  J?+<5)  g  <  fp(x)  +  77  for  x  €  dom/p.  A  parallel  development  gives  identical  results  with  the  roles 
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of  /  and  g  reversed,  where  we  let  gp(x )  =  max{g(x),  —p}  if  x  €  levp  g  n  Bp  and  gp{x)  =  oo  otherwise. 
Specifically,  we  have  that  inf b(x,t)+S)  f  —  9p(x )  +  71  f°r  x  ^  domgp.  Since  x  €  dom/p  if  and  only  if 
x  €  levp  /  n  B p,  we  also  have  that 

inf.B0>}?+<5)  0  <  /pO)  +  V  for  x  <E  levp  f  Cl  Bp. 


and  similarly 

inf B(x,v+S)  f  <  f/pO)  +  rf  for  x  G  levp  g  n  Bp. 

The  lower  bound  then  follows  after  observing  that  these  relations  hold,  in  particular,  for  g  =  dp(f,g). 
Next,  we  address  the  upper  bound.  Suppose  that  g  >  0  satisfies 

infB^)  5  <  max{/(x),  -p}  +  g,  Vx  €  levp  /  n  Bp. 

As  above,  this  means  that  inf^^)  g  <  fp(x)  +  g  for  x  €  dom/p.  We  now  examine  this  relation  for  a 
fixed  x  €  dom/p.  For  every  e  >  0,  there  exists  a  ys  €  A  such  that  d(x, y£)  <  g  and  y(ye)  <  /p  +  g  +  e. 
Set  y0e  =  nra x{y(y£),  fp{x)  -  g  -  e}.  Thus,  g(y£)  <  y0e  and 

fp(x)  ~  Vos  <  fP{x)  -  (fP{x)  ~  V  ~  s)  =  V  +  £■ 

Moreover,  |/p(x)  —  yo£|  <  77  +  e.  We  have  therefore  established  that 

max{d(x,  y£),  \fp(x)  -  y0£|}  <  r\  +  e  and  y(ye)  <  y0e. 

Since  this  holds  for  all  e  >  0, 

inf  |  max{d(x, y),  \fp{x)  -  yo|}  :  g(y)  <  2/o, 2/  €  X,y0  e  Ilj  <  g  for  x  €  dom/p. 

Equivalently,  (x,/p(x))  €  -D+(y)  for  x  €  dom/p.  A  parallel  development  with  the  roles  of  /  and  g 
reversed,  leads  to  (. x,gp(x ))  €  _D+(/)  for  x  €  doiri  gp .  The  implications  established  in  the  beginning 
of  the  proof  show  that  we  then  must  have  that  dp(f,g )  <  g.  In  view  of  the  definition  of  dp(f,g), 
it  is  possible  to  repeat  the  above  arguments  with  g  replaced  by  rf  and  have  gv\d0p(f,g)  as  well  as 
dp(f,g)  <  rf .  This  established  the  upper  bound  of  the  theorem. 

We  next  consider  the  last  assertion  under  the  additional  assumption  that  the  space  is  proper.  Again, 
suppose  that  dp(f,g )  <  77  and,  thus, 

dist((x,  /p(x)),  epi g)  <  g  for  x  €  dom/p  and  dist((x,  gp(x)),  epi /)  <  g  for  x  €  domyp. 

Fix  x  €  dom/p.  By  Lemma  2.4  and  the  fact  that  epiy  is  a  nonempty  closed  set,  there  exists  (y* ,  yg)  E 
Axi?,  with  g(y*)  <  y%,  such  that  g  >  dist((x,  fp(x)),  epiy)  =  d((x,fp(x)),  (y*,yg)).  Hence,  d(x,y*)  <  g 
and  |/p(x)  —  2/q I  —  rl-  which  leads  to  g(y*)  <  fp(x )  +  g  and  y*  €  B(x,  g).  This  fact  and  a  parallel 
development  with  the  roles  of  g  and  /  reversed  give  that 

inf b(xP1)  9  <  /p(^)  +  V  for  x  £  dom/p  and  inf /  <  9p{x)  +  y  for  x  €  domyp. 
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Repeating  the  last  lines  of  reasoning  that  lead  to  the  lower  bound  on  dp(f,g),  we  conclude  that  under 
the  additional  assumption,  the  lower  bound  can  be  improved  to  dp(f,g).  □ 

Proof  of  Proposition  3.3.  We  observe  that  dp(C,  D)  <  oo  since  C,  D  are  nonempty.  Let  p  G  [0,  oo), 
p'  E  (p  +  dp{C ,  D),  oo),  and  e  G  (0 ,  p'  —  p  —  dp{C ,  D)].  Set 


r\e  =  supAp  1/  -  g\  +  max{l,  n{p')} 


cip(C ,  D)  +  e 


First,  we  establish  that 


iniB{x,Ve){g  +  <-£>}  <  max{/(x)  +  tc(x),  -p}  +  %  for  x  G  levp{/  +  ic }  n  Bp. 

Suppose  that  x  G  levp{/  +  lq}  fi  Bp,  which  of  course  implies  that  x  G  C.  There  exists  a  y  G  D  such 
that  d(x,y )  <  inf{d(x,?/)  :  y'  G  D}  +  e.  Thus,  dp(C,D)  >  e(C  (~l  BP,D)  >  dist (x,D)  >  d(x,y )  —  e 
and  d(x,  y )  <  rj£.  Moreover,  d(x,  y)  <  d(x,  x)  +  d(x,  y)  <  p  +  dp(C ,  D)  +  e  <  p'.  These  facts  and  the 
Lipschitz  continuity  of  g  on  Bp/  imply  that 

inf B(x,r,e){9  +  ^d}  <  g{y)  +  tD(y)  =  g{y)  =  g(y )  -  g (x)  +  g(x)  -  f(x)  +  /(x) 

<  K(p')d(x,  y)  +  supAp  |/  -  g\  +  max{/(x)  +  ic{x),  -p} 

<  max{/(x)  +  LC(x),-p}  +  r]e, 

which  establishes  the  first  claim.  Second,  following  a  parallel  argument,  we  realize  that 


inf.B(x,»fe){/  +  Lc)  <  max{p(x)  +  ld(x),  -p}  +  %  for  x  G  1  evp{g  +  iD }  n  Bp. 

Consequently,  dp(f,g )  <  r]E.  Since  this  holds  for  arbitrarily  small  e  >  0,  the  main  conclusion  follows. 
If  (X,  d)  is  proper,  then  the  minimum  distance  between  a  point  and  a  nonempty  closed  set  is  attained 
and  the  above  arguments  hold  with  e  =  0;  see  Lemma  2.4.  This  establishes  that  p'  =  p  +  dp(C,D )  is 
permitted  in  this  case.  □ 


References 

[1]  H.  Attouch.  Variational  Convergence  for  Functions  and  Operators.  Applicable  Mathematics  Sci¬ 
ences.  Pitman,  1984. 

[2]  H.  Attouch,  R.  Lucchetti,  and  R.  J-B  Wets.  The  topology  of  the  p-Hausdorff  distance.  Annali  di 
Matematica  pura  ed  applicata,  CLX:303-320,  1991. 

[3]  H.  Attouch  and  R.  J-B  Wets.  A  convergence  theory  for  saddle  functions.  Transactions  of  the 
American  Mathematical  Society ,  280(1):1-41,  1983. 

[4]  H.  Attouch  and  R.  J-B  Wets.  Isometries  for  the  Legendre-Fenchel  transform.  Transactions  of  the 
American  Mathematical  Society ,  296:33-60,  1986. 


24 


[5]  H.  Attouch  and  R.  J-B  Wets.  Quantitative  stability  of  variational  systems:  I.  The  epigraphical 
distance.  Transactions  of  the  American  Mathematical  Society ,  328(2):695-729,  1991. 

[6]  H.  Attouch  and  R.  J-B  Wets.  Quantitative  stability  of  variational  systems:  II.  A  framework  for 
nonlinear  conditioning.  SIAM  J.  Optimization ,  3:359-381,  1993. 

[7]  H.  Attouch  and  R.  J-B  Wets.  Quantitative  stability  of  variational  systems:  III.  e-approximate 
solutions.  Mathematical  Programming ,  61:197-214,  1993. 

[8]  J.-P.  Aubin  and  H.  Frankowska.  Set-Valued,  Analysis.  Birkhauser,  1990. 

[9]  D.  Aze  and  J.-P.  Penot.  Recent  quantitative  results  about  the  convergence  of  convex  sets  and 
functions.  In  Functional  Analysis  and  Approximations.  Proceedings  of  the  International  Conference 
Bagni  di  Lucca ,  pages  90-110.  Pitagora  Editrice,  1990. 

[10]  G.  Beer.  Topologies  on  Closed  and  Closed  Convex  Sets ,  volume  268  of  Mathematics  and  its  Appli¬ 
cations.  Kluwer,  1992. 

[11]  G.  Beer  and  R.  Lucchetti.  Convex  optimization  and  the  epi-distance  topology.  Transactions  of  the 
American  Mathematical  Society ,  327(2)  :795-813,  1991. 

[12]  G.  Beer  and  R.  Lucchetti.  The  epi-distance  topology:  continuity  and  stability  with  application  to 
convex  optimization.  Mathematics  of  Operations  Research,  17:715-726,  1992. 

[13]  J.  F.  Bonnans  and  A.  Shapiro.  Perturbation  Analysis  of  Optimization  Problems.  Springer,  2000. 

[14]  O.  Bousquet  and  A.  Elisseeff.  Stability  and  generalization.  Journal  of  Machine  Learning  Research, 
2:499-526,  2002. 

[15]  D.  Drusvyatskiy  and  A.  S.  Lewis.  Tilt  stability,  uniform  quadratic  growth,  and  strong  metric 
regularity  of  the  subdifferential.  SIAM  J.  Optimization,  23:256-267,  2013. 

[16]  A.  C.  Eberhard  and  R.  Wenczel.  A  study  of  tilt-stable  optimality  and  sufficient  conditions.  Non¬ 
linear  Analysis,  75:1260-1281,  2012. 

[17]  P.  J.  Laurent.  Approximation  et  optimisation.  Hermann,  1972. 

[18]  A.  S.  Lewis  and  S.  Zhang.  Partial  smoothness,  tilt  stability,  and  generalized  hessians.  SIAM  J. 
Optimization,  23:74-94,  2013. 

[19]  B.  Mordukhovich.  Variational  Analysis  and  Generalized  Differentiation  I:  Basic  Theory. 

Grundlehren  der  mathematischen  Wissenschaften.  Springer,  2  edition,  2013. 

[20]  B.  Mordukhovich.  Variational  Analysis  and  Generalized  Differentiation,  II:  Applications. 

Grundlehren  der  mathematischen  Wissenschaften.  Springer,  2  edition,  2013. 


25 


[21]  B.  S.  Mordukhovich,  R.  T.  Rockafellar,  and  M.  E.  Sarabi.  Characterizations  of  full  stability  in 
constrained  optimization.  SIAM  J.  Optimization,  23:1810-1849,  2013. 

[22]  U.  Mosco.  Convergence  of  convex  sets  and  of  solutions  of  variational  inequalities.  Advances  in 
Mathematics,  3:510-585,  1969. 

[23]  E.  Polak.  Optimization.  Algorithms  and  Consistent  Approximations,  volume  124  of  Applied  Math¬ 
ematical  Sciences.  Springer,  1997. 

[24]  R.T.  Rockafellar  and  R.  J-B  Wets.  Variational  Analysis,  volume  317  of  Grundlehren  der  Mathe- 
matischen  Wissenschaft.  Springer,  3rd  printing-2009  edition,  1998. 

[25]  W.  Romish  and  R.  J-B  Wets.  Stability  of  e-approximate  solutions  to  convex  stochastic  programs. 
SIAM  J.  Optimization,  18(3):961-979,  2007. 

[26]  J.  O.  Royset  and  R.  J-B  Wets.  From  data  to  assessments  and  decisions:  Epi-spline  technology.  In 
A.  Newman,  editor,  INFORMS  Tutorials.  INFORMS,  Catonsville,  2014. 

[27]  J.  O.  Royset  and  R.  J-B  Wets.  Multivariate  epi-splines  and  evolving  function  identification  prob¬ 
lems.  Set-Valued  and  Variational  Analysis,  to  appear,  2016. 

[28]  J.  O.  Royset  and  R.  J-B  Wets.  Optimality  functions  and  lopsided  convergence.  J.  Optimization 
Theory  and  Applications,  to  appear,  2016. 

[29]  J.  O.  Royset  and  R.  J-B  Wets.  Variational  theory  for  optimization  under  stochastic  ambiguity.  In 
review,  2016. 

[30]  J.O.  Royset  and  R.  J-B  Wets.  Fusion  of  hard  and  soft  information  in  nonparametric  density 
estimation.  European  J.  of  Operational  Research ,  247(2)  :532-547,  2015. 

[31]  R.  J-B  Wets.  Convergence  of  convex  functions,  variational  inequalities,  and  convex  optimization 
problems.  In  R.  Cottle,  R.  Giannessi,  and  J.-L.  Lions,  editors,  Varitional  Inequalities  and  Com¬ 
plementary  Problems,  pages  375-403.  Wiley,  New  York,  1980. 

[32]  R.  A.  Wijsman.  Convergence  of  sequences  of  convex  sets,  cones  and  functions.  Bulletin  of  the 
American  Mathematical  Society,  70:186-188,  1964. 

[33]  R.  A.  Wijsman.  Convergence  of  sequences  of  convex  sets,  cones  and  functions.  II.  Transactions  of 
the  American  Mathematical  Society,  123:32-45,  1966. 


26 


